This invention generally relates to processing media content and, more particularly, to a system and related interfaces facilitating the processing of media content.
Recent advances in computing power and related technology have fostered the development of a new generation of powerful software applications. Gaming applications, communications applications, and multimedia applications have particularly benefited from increased processing power and clocking speeds. Indeed, once the province of dedicated, specialty workstations, many personal computing systems now have the capacity to receive, process and render multimedia objects (e.g., audio and video content). While the ability to display (receive, process and render) multimedia content has been around for a while, the ability for a standard computing system to support true multimedia editing applications is relatively new.
In an effort to satisfy this need, Microsoft Corporation introduced an innovative development system supporting advanced user-defined multimedia editing functions. An example of this architecture is presented in U.S. Pat. No. 5,913,038 issued to Griffiths and commonly owned by the assignee of the present invention, the disclosure of which is expressly incorporated herein by reference.
In the ""038 patent, Griffiths introduced the an application program interface which, when exposed to higher-level development applications, enables a user to graphically construct a multimedia processing project by piecing together a collection of xe2x80x9cfiltersxe2x80x9d exposed by the interface. The interface described therein is referred to as a filter graph manager. The filter graph manager controls the data structure of the filter graph and the way data moves through the filter graph. The filter graph manager provides a set of component object model (COM) interfaces for communication between a filter graph and its application. Filters of a filter graph architecture are preferably implemented as COM objects, each implementing one or more interfaces, each of which contains a predefined set of functions, called methods. Methods are called by an application program or other component objects in order to communicate with the object exposing the interface. The application program can also call methods or interfaces exposed by the filter graph manager object.
Filter graphs work with data representing a variety of media (or non-media) data types, each type characterized by a data stream that is processed by the filter components comprising the filter graph. A filter positioned closer to the source of the data is referred to as an upstream filter, while those further down the processing chain is referred to as a downstream filter. For each data stream that the filter handles it exposes at least one virtual pin (i.e., distinguished from a physical pin such as one might find on an integrated circuit). A virtual pin can be implemented as a COM object that represents a point of connection for a unidirectional data stream on a filter. Input pins represent inputs and accept data into the filter, while output pins represent outputs and provide data to other filters. Each of the filters include at least one memory buffer, wherein communication of the media stream between filters is often accomplished by a series of xe2x80x9ccopyxe2x80x9d operations from one filter to another.
As introduced in Griffiths, a filter graph has three different types of filters: source filters, transform filters, and rendering filters. A source filter is used to load data from some source; a transform filter processes and passes data; and a rendering filter renders data to a hardware device or other locations (e.g., saved to a file, etc.). An example of a filter graph for a simplistic media rendering process is presented with reference to FIG. 1.
FIG. 1 graphically illustrates an example filter graph for rendering media content. As shown, the filter graph 100 is comprised of a plurality of filters 102-114, which read, process (transform) and render media content from a selected source file. As shown, the filter graph includes each of the types of filters described above, interconnected in a linear fashion.
Products utilizing the filter graph have been well received in the market as it has opened the door to multimedia editing using otherwise standard computing systems. It is to be appreciated, however, that the construction and implementation of the filter graphs are computationally intensive and expensive in terms of memory usage. Even the most simple of filter graphs requires and abundance of memory to facilitate the copy operations required to move data between filters. Complex filter graphs can become unwieldy, due in part to the linear nature of prior art filter graph architecture. Moreover, it is to be appreciated that the filter graphs themselves consume memory resources, thereby compounding the issue introduced above.
Thus, what is required is a filter graph architecture which reduces the computational and memory resources required to support even the most complex of multimedia projects. Indeed, what is required is a dynamically reconfigurable multimedia editing system and related methods, unencumbered by the limitations described above. Just such a system and methods are disclosed below.
Methods and systems for processing media content are described. In one embodiment, a software-implemented matrix switch is used in connection with multi-media editing software that enables users build or define their own editing multi-media editing projects. Editing projects typically incorporate multiple different user-selected clips into an integrated project. Each clip can be defined in terms of one or more digital data streams, e.g. video and audio streams. In this context, one implementation provides a software-implemented method that defines a first data structure that represents a user-defined multi-media editing project. The first data structure is processed to provide a second data structure that contains data that can be used to program the matrix switch so that multiple switch inputs are routed to multiple switch outputs. The second data structure is operated upon so that no two inputs are routed to a particular single output of the matrix switch during a common time period. In one embodiment, added flexibility and efficiency is provided through the use of a composite or composition. A composite or composition can be considered as a representation of an editing project as a single track. Editing projects can have one or more tracks, and each track can be associated with one or more data stream sources that can have effects or transitions applied on them. Compositions can be nested inside one another and provide an economical way for complex editing manipulations to take place.
The methods and systems permit the programmed matrix switch to receive multiple different digital data streams and route certain streams to certain switch outputs. The routed streams are processed, e.g. combined, mixed, transitioned, and the like, by software elements that are incorporated into one or more feedback loops that connect with one or more respective inputs of the matrix switch. A routing scheme derived from a data structure ensures that the appropriate data stream is routed to the appropriate switch output at the right time.